feat(logging): add support for key-value pairs in LambdaJsonEncoder#2377
feat(logging): add support for key-value pairs in LambdaJsonEncoder#2377cmtjk wants to merge 1 commit intoaws-powertools:mainfrom
Conversation
|
We've decided to permit The only issue we see is that the string I'll provide updated code later in my free time. String s = null;
log.atInfo().setMessage("Processing event")
.addKeyValue(null, s)
.addKeyValue("eventId", 54321)
.addKeyValue("eventId", "abcde")
.addKeyValue(null, "fghij")
.addKeyValue("klmnop", s)
.addKeyValue("qrstu", "true")
.addKeyValue("vwxyz", false)
.log();{
"level": "INFO",
"message": "Processing event",
"cold_start": false,
"function_arn": null,
"function_memory_size": 0,
"function_name": null,
"function_request_id": "null",
"function_version": null,
"service": "service_undefined",
"kvpList": [
{
"null": "null"
},
{
"eventId": 54321
},
{
"eventId": "abcde"
},
{
"null": "fghij"
},
{
"klmnop": "null"
},
{
"qrstu": true
},
{
"vwxyz": false
}
],
"timestamp": "2026-02-03T08:17:13.464Z"
} |
There was a problem hiding this comment.
Hey @cmtjk! Awesome work so far. I think adding this feature will be beneficial for the open-source community and a valuable contribution to the project. Here are some of my thoughts:
- We need to add the same behavior to the Log4j2 backend and the LambdaEcsEncoder in the logback backend.
- We need to add thorough tests. E.g. What happens if I set a key via fluent API that clashes with a previously set MDC key? I think it is ok to overwrite for that log statement and document this behavior in the docs.
- We should update the E2E tests with an SLF4J fluent API test. https://github.com/cmtjk/powertools-lambda-java/blob/improv/logback-logger-key-value-pairs/powertools-e2e-tests/src/test/java/software/amazon/lambda/powertools/LoggingE2ET.java#L73
The only issue we see is that the string "true" is currently serialized as the boolean true, which is often equivalent but not always correct.
Good catch. We should fix and make sure it is consistent for both MDC and fluent API keys.
| kv -> String.valueOf(kv.key), | ||
| kv -> String.valueOf(kv.value), | ||
| (existingKey, newKey) -> newKey, | ||
| HashMap::new |
There was a problem hiding this comment.
Let's use a TreeMap to be consistent with the MDC fields.
| // TODO: Existing keys are overwritten, need to handle these cases | ||
| // new KeyValuePair(null, null))); | ||
| // new KeyValuePair("", "")); |
There was a problem hiding this comment.
Good point. I think it is okay for now to overwrite with the latest added key. We have a warning in the docs about this: https://docs.aws.amazon.com/powertools/java/latest/core/logging/#custom-keys.
The same will happen, for example, if a user sets a reserved powertools key via MDC.
Avoid adding any of the keys listed in standard structured keys and additional structured keys to your MDC. This may cause unindented behavior and will overwrite the context set by the Logger. Unlike with StructuredArguments, the Logger will not ignore reserved keys set via MDC.
|
Hi @phipag, I just want to confirm that we should drop the solution mentioned in #2377 (comment) and instead add proper documentation. |
Yes let's do it. Unless you have any strong arguments against it? But I agree that this would be what the user expects. |
cmtjk
force-pushed
the
improv/logback-logger-key-value-pairs
branch
from
31c8a93 to
2b4b6a3
Compare
|
2 participants
Summary
Key-value pairs added via SLF4J's fluent API, i.e.
org.slf4j.spi.LoggingEventBuilder::addKeyValue(String var1, Object var2), are ignored bysoftware.amazon.lambda.powertools.logging.logback.LambdaJsonEncoder.This PR aims to properly encode these key-value pairs.
Changes
Serialize
org.slf4j.event.LoggingEvent::getKeyValuePairs()like MDC entries, overriding duplicate keys (see below).Discussion number: 2375
WIP
Key-value handling needs clarification. The current API stores key-value pairs in separate collections and allows duplicate keys, so it’s unclear whether the encoder should preserve duplicates or merge them. For reference, the
ch.qos.logback.classic.encoder.JsonEncoderencodes key-value pairs as an array of{"key": "value"}entries to preserve duplicates.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.